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I. Real Party in Interest 

The real parties in interest are the inventors: Joe Said and David Schleppenbach. 



3 



10/686,127 



Appeal Brief 



II. Related Appeals and Interferences 

Appellants, Appellants' legal representative, and Assignee know of no other prior or 
pending appeals, interferences or judicial proceedings that may be related to, directly affect or be 
directly affected by or have a bearing on the Board's decision in the pending appeal 
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III. Status of Claims 

Pending: Claims 1-4, 6-12 and 14-22 

Canceled: 5 and 13 

Allowed: None 

Rejected: Claims 1-4, 6-12 and 14-22 

Appealed: Claims 1 -4, 6- 1 2 and 1 4-22 
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IV. Status of Amendments 

An amendment submitted on August 21, 2007 prior to the issuance of final rejection of 
Claims 1-4, 6-12 and 14-22 was entered by the Examiner. An Amendment After Final was filed 
on January 22, 2008 and was entered by the Examiner. No further amendments have been 
submitted. 
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V. Summary of Claimed Subject Matter 

The following explanation of the subject matter defined in each of the independent claims 
at issue in this appeal is provided with reference to page and line numbers in the specification 
and to reference characters in the drawings in compliance with 37 C.F.R. §4L37(c)(v). These 
references are made to exemplary embodiments disclosed in the application in an effort to enable 
the Board to more quickly determine where the claimed subject matter is described in the 
application, and do not limit the scope of the claims. 

A. Independent Claim 1 

The subject matter defined in independent claim 1 relates to a method of input conversion 
to output, including entering a first stage input 102 (FIG. 1) and a second input 104 (paragraph 
[0056] of the present publication). The first input and the second input are translated into an 
electronic format 106 (paragraph [0057]). The electronic format is converted into standard XML 
108 encoded with accessibility information (paragraph [0060]). The standard XML encoded 
with accessibility information is transformed into individual version of XML 110 dependent on 
desired output (paragraphs [0061] and [0062]). A rendering engine 1 12 is utilized to modify the 
individual version of XML into a format for the output (paragraph [0063]). 

B. Independent Claim 19 

The subject matter defined in independent claim 19 relates to a method of input 
conversion to output, including entering a first input 102 (FIG. 1) including verbal information 
and a second input 104 including non-verbal information (paragraphs [0056] and [0060] of the 
present publication). The second input is entered in parallel with the first input and is entered via 
a same medium as the first input. The same medium being one of a visual medium and an audial 
medium (paragraph [0056]). The first input and the second input are translated into an electronic ~ 
format 106 (paragraph [0057]). The electronic format is converted into standard XML 108 
encoded with accessibility information (paragraph [0060]). The standard XML encoded with 
accessibility information is transformed using extensible stylesheet language transformations 
into XHTML 1 10 for desired output of text (paragraphs [0061] and [0062]). A rendering engine 
1 12 of accessible instant messenger is utilized to modify the XHTML into the desired output of 
text (paragraphs [0063 and [0064]). 

C. Independent Claim 20 

The subject matter defined in independent claim 20 relates to a method of input 
conversion to output, including entering a first input 102 (FIG. 1) including text information and 
a second input 104 including semantic information (paragraphs [0056] and [0059] of the present 
publication). The first input and the second input are translated into an electronic format 106 
(paragraph [0057]). The electronic format is converted into standard XML 108 encoded with 
accessibility information (paragraph [0060]). The standard XML encoded with accessibility 
information is transformed into individual version of XML 110 dependent on desired output 
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(paragraphs [0061] and [0062]). A rendering engine 1 12 is utilized to modify the individual 
version of XML into a format for the output (paragraph [0063]). 
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VI. Grounds of Rejection to be Reviewed on Appeal 

A. Claims 8, 9, 21 and 22 

Claims 8, 9, 21 and 22 stand rejected under 35 U.S.C. §112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
Appellants regard as the invention. 

B. Claims 1-17 and 19-22 

Claims 1-17 and 19-22 stand rejected under 35 U.S.C. §103(a) as being unpatentable over 
U.S. Patent Publication No. 2003/0130854 (Galanes). 

C. Claim 18 

Claim 18 stands rejected under 35 U.S.C. 103(a) as being unpatentable over Galanes in 
view of Lemlouma ("NAC: A Basic Core for the Adaptation and Negotiation of Multimedia 
Services" 2001). 
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VII. Argument 

A. Rejection of Claims 8, 9, 21 and 22 under 35 U.S.C. $112. second paragraph 

Claims 8, 9, 21 and 22 were rejected under 35 U.S.C. §112, second paragraph as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
Appellants regard as the invention. 

1. Claim 8 is in Allowable Form 

Claim 8 recites in part that the "translating step includes the use of automatic speech 
recognition of the first input and the second input to produce the electronic format, the first input 
including verbal information, and the second input including non-verbal information." The 
Examiner states that "this presents a logical error. The second input, being non-verbal input, 
cannot be recognized by an ASR system." Appellants respectfully point out, however, that claim 
8 does not recite that the second input is a non-verbal input, as stated by the Examiner. Rather, 
claim 8 recites "the second input including non-verbal information ." {Emphasis Added). Such 
subject matter is disclosed in paragraph [0059] of the present publication wherein, discussing the 
input associated with sub-process I y , it is stated that "a blind user could speak the word 'smiley- 
face' to embed a smiley-face © emoticon." Thus, this second input in the form of the spoken 
word "smiley- face" may be translated via automatic speech recognition and at the same time 
includes non-verbal information in the form of a smiley-face © emoticon. Thus, claim 8 
includes no logical error. Of course a smiley- face © emoticon is just one example of such non- 
verbal information, and the scope of claim 8 is not to be limited to non-verbal information being 
in the form of a smiley-face © emoticon. 

For all of the foregoing reasons, Appellants respectfully submit that claim 8 is in 
allowable form. 

2, Claim 9 is in Allowable Form 

Claim 9 recites in part that the "translating step includes the use of sign language 
recognition of the first input and the second input to produce the electronic format, the first input 
including verbal information, and the second input including non-verbal information." The 
Examiner states that "this also produces a logical error. A sign language recognition system is 
capable of recognizing gestures alone, not speech." Appellants respectfully submit, however, 
that the Examiner is mistakenly assuming that verbal information can be conveyed via speech 
only. In fact, it is also possible to convey verbal information via gestures and facial expressions. 
This is specifically disclosed at paragraph [0056] of the present publication in which it is stated 
that the user may enter raw text information by "gesturing (sign language)" as well as by 
speaking. Text information may be equated to verbal information, as the present publication 
expressly makes clear in paragraph [0060] wherein it is disclosed that a user may "communicate 
both verbal (text) and non-verbal (semantic) information as part of the input." {Emphasis Added). 
Further, in paragraph [0058 it is disclosed that "the special SLR (sign-language recognition) 
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engine is used to convert gestures and facial expressions into a text string." Thus, a SLR engine 
may convert gestures into text (verbal) information. 

For all of the foregoing reasons, Appellants respectfully submit that claim 9 includes no 
logical error and is in allowable form. 

3. Claim 21 is in Allowable Form 

Claim 21 was rejected for substantially the same reasons that claim 8 was rejected. For 
all of the reasons provided above with regard to claim 8, Appellants respectfully submit that 
claim 21 is also in allowable form. 

4. Claim 22 is in Allowable Form 

Claim 22 was rejected for substantially the same reasons that claim 9 was rejected. For 
all of the reasons provided above with regard to claim 9, Appellants respectfully submit that 
claim 22 is also in allowable form. 



B. Rejection of Claims 1-17 and 19-22 under 35 U.S.C §103(a) 

Claims 1-17 and 19-22 were rejected under 35 U.S.C. §103(a) as being unpatentable over 
U.S. Patent Publication No. 2003/0130854 (Galanes). 

L Independent Claim 1 is Patentable 

Claim 1 recites in part "entering a first input and a second input; translating the first input 
and the second input into an electronic format; converting the electronic format into' standard ~ ' 
XML encoded with accessibility information ". (Emphasis added). Appellants respectfully 
submit that such limitations are neither taught, disclosed nor suggested by Galanes and include 
distinct advantages thereover. 

The Examiner cites [0049] of Galanes disclosing that "the XML is extended to include 
multimodal recognition functions (accessibility information)" However, multimodal recognition 
functions are not accessibility information. Accessibility information is non-textual, semantic 
information that indicates non-verbal aspects of the textual communication, such as tone of voice 
and emotional content, for example (paragraphs [0023] and [0060] of the present publication). 
Recognition of different modes of operation is unrelated to semantic or non-verbal aspects of any 
particular text communication. That is, Galanes is completely silent as to any of the modes such 
as handwriting, speech, gestures, etc. being used to communicate accessibility information. 
Although in paragraph [0050] of Galanes it is disclosed that extensions or "tags" may have 
semantics, Galanes is completely silent as to these semantics indicating non-verbal aspects of the 
textual communication, such as tone of voice and emotional content, for example. Thus, these 
semantics are not accessibility information. 
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At page 2 of the Final Office Action dated 1 1/20/2007, the Examiner states that: 

[i]n response to Appellant's argument that the references fail to show certain 
features of Appellant's invention, it is noted that the features upon which 
Appellant relies (i.e., that Accessibility information is non-textual, semantic 
information that indicates non-verbal aspects of the textual communication, such 
as tone of voice and emotional content, for example (Remarks page 9)) are not 
recited in the rejected claim(s). Although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. 

Appellants respectfully submit, however, that Appellants' definition of "accessibility 
information" is not an additional limitation from the specification that is being read into the 
claims, but rather is simply a definition of a claim term that is consistent with how the term is 
used in the specification. "An Appellant is entitled to be his or her own lexicographer and may 
rebut the presumption that claim terms are to be given their ordinary and customary meaning by 
clearing setting forth a definition of the term that is different from its ordinary and customary 
meaning(s)." MPEP §2111.01 IV., quoting In re Paulsen, 30 F. 3d 1475, 1489, 31 USPQ2d 1671, 
1674 (Fed. Cir. 1994). Appellants submit that the term "accessibility information" does not have 
any ordinary or customary meaning either in the art or in common usage, and thus there is no 
presumed definition to be rebutted. 

Further, "[t]he specification should also be relied on for more than just explicit 
lexicography or clear disavowal of claim scope to determine the meaning of a claim term when 
Appellant acts as his or her own lexicographer; the meaning of a particular claim term may be 
defined by implication, that is, according to the usage of the term in the context in the 
specification." Id, quoting Phillips v. AWH Corp., 415 F3d 1303, 75 USPQ2d 1321 (Fed Cir. 
2005)- (en banc): Appellants submit that Appellants' proposed definition of "accessibility * ' ' ~ 
information" (i.e., non-textual, semantic information that indicates non-verbal aspects of the 
textual communication, such as tone of voice and emotional content, for example) is at least 
implicitly defined in paragraphs [0023] and [0060] of the present publication, and thus should be 
accepted. 

For all of the above reasons, Galanes does not disclose or suggest entering a first input 
and a second input, translating the first input and the second input into an electronic format, and 
converting the electronic format into standard XML encoded with accessibility information, as 
recited by amended claim 1 . 

The present invention includes distinct advantages over Galanes. By translating two 
inputs into an electronic format and converting the electronic format into standard XML encoded 
with accessibility information, it is possible for users with disabilities to communicate non- 
verbal aspects of textual messages. For example, one of the inputs may include textual 
information, and the other input may include information pertaining to tone of voice, emotional 
content, points of emphasis, etc. 
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For all of the above reason, claims 1, and claims 2-4, 6-12 and 14-17 depending 
therefrom, are in condition for allowance. Claims 5 and 13 were canceled in the Amendment 
filed August 21, 2007. 

2. Claim 6 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 6 is separately 
patentable. Claim 6 recites in part that the "entering step includes semantic information." 
(Emphasis added). Appellant respectfully submits that such limitations are neither taught, 
disclosed nor suggested by Galanes and include distinct advantages thereover. 

The Examiner cites paragraph [0027] of Galanes as disclosing entering of spoken input. 
However, spoken inputs generally do not include semantic information. Semantic information, 
as defined by the present publication (paragraphs [0059] and [0060]), is non-verbal information 
that may add emotional content or emphasis to verbal content, for example. The American 
Heritage Dictionary defines "semantic*' as "[o]f or relating to meaning, especially meaning in 
language." Thus, semantic information may enhance the meaning of verbal content. For 
example, a blind user could speak the word "smiley-face" to embed a smiley- face emoticon 
(paragraph [0059]). Thus, Galanes does not disclose or suggest an entering step including 
semantic information, as recited by claim 6. For the above reasons, claim 6 is in condition for 
allowance. 

3» Claim 7 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 7 is separately 
patentable. Claim 7 recites in part that the "entering step includes providing formatting and 
- structure for text ." (Emphasis added). Such-subject matteris disclosed at paragraph [0014] of < 
the present specification. Appellants respectfully submit that such limitations are neither taught, 
disclosed nor suggested by Galanes and include distinct advantages thereover. 

The Examiner cites Galanes as disclosing a pointing device being used to indicate fields 
for data entry. Appellants submit that fields for data entry are not formatting and structure for 
text. Thus, Galanes does not disclose or suggest that an entering step includes providing 
formatting and structure for text, as recited by claim 7. For the above reasons, claim 7 is in 
condition for allowance. 

3. Claim 8 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 8 is separately 
patentable. Claim 8 recites in part that the "translating step includes the use of automatic speech 
recognition of the first input and the second input to produce the electronic format, the first input 
including verbal information, and the second input including non-verbal information." 
(Emphasis added). Such subject matter is disclosed at paragraphs [0059] and [0060] of the 
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present publication. Appellant respectfully submits that such limitations are neither taught, 
disclosed nor suggested by Galanes and include distinct advantages thereover. 

The Examiner cites Galanes as disclosing speech input in conjunction with multiple 
multimodal inputs. However multimodal inputs are not spoken inputs on which automatic 
speech recognition is used. Galanes is completely silent as to a first spoken input including 
verbal information and a second spoken input including non-verbal information. Thus, Galanes 
does not disclose or suggest that a translating step includes the use of automatic speech 
recognition of a first input and a second input to produce an electronic format, the first input 
including verbal information, and the second input including non-verbal information, as recited 
by claim 8. For the above reasons, claim 8 is in condition for allowance. 

4. Claim 9 is Separately Patentable 

In addition to being allowable by virtue of depending from claim 1, claim 9 is separately 
patentable. Claim 9 recites in part that the "translating step includes the use of sign language 
recognition of the first input and the second input to produce the electronic format, the first input 
including verbal information, and the second input including non-verbal information." 
(Emphasis added). Such subject matter is disclosed at paragraphs [0059] and [0060] of the 
present publication. Appellant respectfully submits that such limitations are neither taught, 
disclosed nor suggested by Galanes and include distinct advantages thereover. 

The Examiner cites Galanes as disclosing speech input in conjunction with multiple 
multimodal inputs. However, neither speech inputs nor multimodal inputs are gestured inputs on 
which sign language recognition is used. Galanes is completely silent as to a first gestured input 
including verbal information and a second gestured input including non-verbal information. 
Thus, Galanes not teach, disclose or suggest that a translating step includes the use of sign 
-language-recognition of a first input and a second input to produce an electronic format, the firsr* 
input including verbal information, and the second input including non-verbal information, as 
recited by claim 9. For the above reasons, claim 9 is in condition for allowance. 

5. Independent Claim 19 is Patentable 

Claim 19 recites in part "entering a first input including verbal information and a second 
input including non-verbal information, the second input being entered in parallel with the first 
input, the second input being entered via a same medium as the first input, the same medium 
being one of a visual medium and an audial medium ". (Emphasis added). A deaf user, for 
example, could enter the verbal and non-verbal inputs in parallel via a same visual medium such 
as sign language recognition. As another example, a blind user could enter the verbal and non- 
verbal inputs in parallel via a same audial medium such as automatic speech recognition. 

Galanes is completely silent as to verbal and non-verbal inputs being entered via a same 
visual or audial medium. The Examiner cites Galanes as disclosing a computer receiving speech 
and information entered through a keyboard. However, Appellants respectfully submit that 
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speech and keyboard entry are not a same medium, as required by claim 19. Moreover, 
keyboard entry is neither a visual medium nor an audial medium, as also is required by claim 19. 
Thus, Galanes does not disclose or suggest a second input being entered in parallel with a first 
input, the second input being entered via a same medium as the first input, the same medium 
being a visual medium or an audial medium, as recited by claim 19. 

The present invention as recited by claim 19 includes distinct advantages over Galanes. 
A user with a disability may be able to provide inputs via only a single type of medium. For 
example, a blind user may be able to enter inputs though only an audial medium such as speech 
recognition. Thus, the present invention enables disabled users to enter both verbal and non- 
verbal information in parallel (i.e., as the user conceptualizes the verbal and non-verbal 
information in conjunction with each other) via a same single medium that may be the only 
medium that the disabled user may effectively use. 

Moreover, claim 19 also recites in part "converting the electronic format into standard 
XML encoded with accessibility information ". (Emphasis added). Thus, claim 19 also recites 
subject matter substantially similar to the subject matter recited by claim 1. Hence, claim 19 is 
also in condition for allowance for all of the reasons given above with regard to claim 1 . 

For the above reasons, claim 19, and claims 21 and 22 depending therefrom, are in 
condition for allowance. 

6. Claim 21 is Separately Patentable 

Claim 21, depending from claim 19, recites that "the same medium comprises automatic 
speech recognition ." (Emphasis added). Thus, both verbal and non-verbal information are 
entered in parallel via speech recognition. The Examiner cites Galanes as disclosing a computer 
— receiving speech and information entered through a keyboard: However;* Appellants 1 respectfully ~ 
submit that keyboard entry is not automatic speech recognition, as required by claim 21 . 
Appellants submit that claim 21 is in condition for allowance. 

7. Claim 22 is Separately Patentable 

Claim 22, depending from claim 19, recites that "the same medium comprises sign 
language recognition ." (Emphasis added). Thus, both verbal and non-verbal information are 
entered in parallel via sign language recognition. The Examiner cites Galanes as disclosing a 
computer receiving speech and information entered through a keyboard. However, Appellants 
respectfully submit that neither speech recognition nor keyboard entry is sign language 
recognition, as required by claim 22. Appellants submit that claim 22 is in condition for 
allowance. 
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8, Independent Claim 20 is Patentable 

Claim 20 recites "entering a first input including text information and a second input 
including semantic information ; translating the first input and the second input into an electronic 
format; converting the electronic format into standard XML encoded with accessibility 
information /' (Emphasis added). The Examiner cites Galanes as disclosing speech as semantic 
information. However, Appellants respectfully submit that speech is not semantic (non-verbal) 
information as "semantic information" is defined in paragraphs [0059] and [0060] of the present 
publication. Galanes is completely silent as to entering a first input including text information 
and a second input including semantic information. As discussed above with regard to claim 6, 
the semantic information may enhance the meaning of the text information with emotional 
content or points of emphasis, for example. Moreover, as discussed above with regard to claim 1, 
Galanes does not disclose converting an electronic format into standard XML encoded with 
accessibility information, as also required by claim 20. 

C. Rejection of Claim 18 under 35 U.S.C §103(a) 

Claim 18 was rejected under 35 U.S.C. § 103(a) as being unpatentable over Galanes in 
view of Lemlouma. However, claim 18 depends from claim 1, which is in condition for 
allowance for all of the reasons given above. Accordingly, claim 18 is also in condition for 
allowance. 

In view of the remarks above, Appellants respectfully submit that the present invention is 
in condition for allowance and respectfully request the Honorable Board of Appeals to reverse 
the Examiner's rejections from the Final Office Action. 



^ Respectfully submitted, 



April 2 L 2008 
Date 




Reg. No. 43,176 

Sommer Barnard PC 

One Indiana Square, Suite 3500 

Indianapolis, IN 46204 

(317)713-3500 (phone) 

(317)713-3699 (fax) 

kswedo@sommerbarnard.com 
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VIII. Claims Appendix 

List of Claims 

1 . A method of input conversion to output, said conversion method comprising: 
entering a first stage input and a second input; 

translating the first input and the second input into an electronic format; 
converting the electronic format into standard XML encoded with accessibility s 
information; 

transforming the standard XML encoded with accessibility information into individual 
version of XML dependent on desired output; and 

utilizing a rendering engine to modify the individual version of XML into a format for 
the output. 

2. The conversion method of claim 1 wherein said entering step includes speaking. 

3. The conversion method of claim 1 wherein said entering step includes gesturing. 

4. The conversion method of claim 1 wherein said entering step includes writing. 

5. (Canceled) 

6. The conversion method of claim 1 wherein said entering step includes semantic information. 
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7. The conversion method of claim 1 wherein said entering step includes providing formatting 
and structure for text. 

8. The conversion method of claim 2 wherein said translating step includes the use of automatic 
speech recognition of the first input and the second input to produce the electronic format, the 
first input including verbal information, and the second input including non-verbal information. 

9. The conversion method of claim 3 wherein said translating step includes the use of sign 
language recognition of the first input and the second input to produce the electronic format, the 
first input including verbal information, and the second input including non-verbal information. 

10. The conversion method of claim 1 wherein the individual version of XML is XHTML for 
desired output. 

11. The conversion method of claim 1 wherein the individual version of XML is VoiceXML for 
desired output. 

12. The conversion method of claim 1 wherein the accessibility information includes non-verbal 
information. 

13. (Canceled) 
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14. The conversion method of claim 1 1 wherein the desired output is synthesized speech. 

15. The conversion method of claim 12 wherein the desired output is virtual sign language. 

16. The conversion method of claim 12 wherein the desired output is electronic braille. 

1 7. The conversion method of claim 12 wherein the desired output is electronic large print. 

18. The conversion method of claim 1 wherein said transforming step includes the use of 
extensible stylesheet language transformations to transform the standard XML encoded with 
accessibility information into the individual version of XML dependent on desired output. 
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19. A method of input conversion to output, said conversion method comprising: 

entering a first input including verbal information and a second input including non- 
verbal information, the second input being entered in parallel with the first input, the second 
input being entered via a same medium as the first input, the same medium being one of a visual 
medium and an audial medium; 

translating the first input and the second input into an electronic format; 
converting the electronic format into standard XML encoded with accessibility 
information; 

transforming the standard XML encoded with accessibility information using extensible 
stylesheet language transformations into XHTML for desired output of text; and 

utilizing a rendering engine of accessible instant messenger to modify the XHTML into 
the desired output of text. 
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20. A method of input conversion to output, said conversion method comprising: 

entering a first input including text information and a second input including semantic 
information; 

translating the first input and the second input into an electronic format; 
converting the electronic format into standard XML encoded with accessibility 
information; 

transforming the standard XML encoded with accessibility information into individual 
version of XML dependent on desired output; and 

utilizing a rendering engine to modify the individual version of XML into a format for 
the output. 

21. The conversion method of claim 19 wherein the same medium comprises automatic speech 
recognition. 

22. The conversion method of claim 19 wherein the same medium comprises sign language 
recognition. 
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IX. Evidence Appendix 

No evidence has been entered. 
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X. Related Proceedings Appendix 

Appellants are unaware of any Related Proceedings. 
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